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"We will go to war again and we will spend money, so we need to make sure we are spending money 
better and more efficiently. " - Interviewee A 


Description and Methodology 


OHHH most directly to the research you [SIGAR] are doing and have really gained momentum 
over the last three or four years. We, as a contractor, conduct them on an annual basis. It is a worldwide list 
with each country having an established rank based on fragility and instability. Each list is based on an 
analytic framework. We cyclically analyze the data which is updated each year to create the scores and then 
organize them into the formal list. 


A) What is unique is that we u se a comprehensive set of open sources to generate the data. We used to use 
|. They are split into four categories: political, security, economic, or social. 


(b)(3), (b)(6). (b)(7)(C) 
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Below these four levels, the subdivision then looks at components like legitimacy or efficiency. We use 
statistical methods to rank the countries and conduct the analysis. The portion dedicated to instability is 
more of a forecasting methodology and is only five or six indicators which we pull from the state failure 
indicators. Our main concern is what has happened prior and is less focused on predictive analysis. 


Our latest contract included an updated methodology. To test this methodology we have to retroactively 
predict the Arab Spring; we had to predict the rising insecurity. 


C) It [the methodology] has been revised substantially and we will finalize it 
weeks. Once finalized, we will proceed with working with this new model. 


(bXJ). 0>X6). (bX'XC) 


in two or three 


A) Bothare at the country level unit of analysis. Analysis at this level, 
however, can’t say when or where to ce rtain degrees. It can't look at which administrative units are more 
or less prone to instability. The intent program was to get granular, and, for 

example, focus on development aid, intra-state conflict and how to capitalize on available conflict data 
(which is usually geo-coded). There is emphasis on Africa, since there is so much data right now, but also in 
other countries like Afghanistan. With the geo-coded conflict data we can look at similarly geo-coded aid 
projects and how they may overlap or conflict - temporally or geographically. We tend not to worry about 
getting down to resolution at the daily level, but instead look at the quarter or half-year levels and at least 
at the first order administrative divisions instead of the country-level. 


—Following text in this section is from later in the conversation but fit best here — 

C)HMW actuall y 8 ave us a brief history of the|jjj|j||^| not too lo ng^aga T he first list was in about 
2005 or 2006 and was met with universal hostility inside and outside 0 fj||j^j|| 

A) The tension was between the office oand other internal 
offices. The field offices believed their co untry was different and that they are the ones who truly know its 
history and the important variables. The|j^|j|j^j|f however, was still a systematic way to measure many 
countries and to measure variations across many countries. The rankings were hard because it would 
compare one country against another, and staff at launch events for the list still maintained that their 
country was different. 


C) In the last three or four years, the level of institutional appreciation has increased substantially. There 
has also been a steady increase in interest from the interagency. The lists were usually used internally but 
now are spreading through the interagency and getting recognition. People want it and the audience is 
more receptive. 


A) It [the improved recejptionofp^|^^^j^y is correlated He was formerly with our 

group and is now with|m|i S an experienced advocate and pushes it forward internally quite well. 

C^| is also a PhD. in political science and can convey the details of the list and many levels of expertise. 

AlW^iav^beenucon^ctec^^eople all over the world who including 

and many defense and interior ministries who use the^|for planning 
purposes. They|||j|£j|jalso use it for risk forecasting and to plan around what countries are at most risk. 


C) "It is my view that the U.S. Government is more receptive to evidence-based studies even more so 
now since budgets are so tight and need to lean toward this type of analysis [to be more efficient with 
spending ]," 


Chan g ing Nature of Conflict Ana lysis 

C) One aspect [of how the study of intra-state conflict] has changed over the last fourteen years is the 
academic literature. The academic literature concerning the onset of civil war or intra-state conflict was 
very much focused on the country-level and we used largely structural variables like GDP per capita or 
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ethnic heterogeneity. For assessing conflict probability we looked at regime type and status, and regime 
orientation to the international economy. 

A) It was this level of anal ysis [on reg ime characteristics or GDP per capita] that were reflected in the 
instability analysis for only because that was what data existed and there thus a rationale for 

using it. 

C) There was a real explosion of data that happened and it disaggregated conflict data and usually geo- 
coded it; there is a lot available now on predictive variables. The basic thing I did was to identify actors that 
challenged the regime, such as rebel combatants, and their characteristics. There was a lot of data on ethnic 
exclusion through our Zurich connection [International Conflict Research group at ETH Zurich] . We 
disaggregated data below the country-level to the local level and could look at where ethnic groups are and 
their relationship to the regime and inter-ethnic dynamics. 

A] I see the evolution of the data a bit like the field of chemistry. For decades we thought the atom was the 
smallest level of analysis, but now we know we can go even deeper. Now the country level analysis is too 
limiting and we break the data in smaller units like rebel groups. 

Current Applications 

C) We now have data on rebel groups and how they conflict with each other. We can generate data on the 
size of "shelter” armed forces provide rebel groups. Now we have a substantial amount of data that allows 
for finer grain theory testing (spatially and temporally). 

A) As for the physical presence variables, interviewee 'b' has been looking at that. He looks at number and 
location of bases and the strength of armed forces. Others are doing similar analysis on territorial control 
versus where conflict has occurred. Now we can turn the geocodes or the physical location of conflict into a 
map of where is it happening and see what else is happening in the same area (such as the ethnic 
landscape, economic activity or aid projects). In the last two or three years we have also seen the 
increased use of satellite data to measure economic productivity. It was hard to measure economic activity 
until now with the use of satellites. 

C) We used to have to do cross-sectional analysis and the'cross-national level. 

A) Now we can look dynamically at more countries; not just at one variable as it was five years ago, but 
now we have more than a dozen of variables with this resolution [on economic activity]. 

C) We also are now looking at data regarding the killing of the civilian population by the government and 
the rebel groups - "one-sided attacks". The Uppsala Conflict Data Program (UCDP) in Sweden is doing a lot 
to improve this data set. We can now ask increasingly sophisticated questions on when and why civilians 
are targeted. We look also at where and have even disaggregated variables as to why. 

B) The Empirical Studies of Conflict project (ESOC) took event level data and ran an analysis, but our 
concern was how they aggregated the total number of SIGACTs. 

—Following text in this section is from later in the conversation but fit best here — 

A) The evolution of the field and the methodology used to study conflict at the country level broaden 
beyond what was largely econometrics. We can now do analysis at lower level landscapes wi th 
computational modeling and can start to peer into counterfactuaIs.[ffi ft9MIDIG)MtDim@E is doing some 
work on Afghanistan and may have examples of computational modeling. The advantage is the ability to 
look at counterfactuals such as what if the distribution or deployment of money or forces was different and 
the potential affects. The issue was always what do we evaluate against? What would an alternative past 
look like? 
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C) Scenario analysis is also more interesting with computational modeling because you can move beyond 
the 'toy models' and start adding is hard evidence and contextualizing. If you change certain pieces of the 
model, your faith increases and so does the model’s credibility. 

A) The re were really only two scholarly articles on computational modeling published around 2001. One 

was which looked at the formation of post-Soviet states and 

one [0lQ iBgilpjl^ffiffiSB|B^^dinthej ournal of C onflict Resolution w hich focused on ethnic 

were contacted by m any government agencies fi 

| concerning our work on leadership structures and closed regimes. Today,^| 
jjhaspublished has published more than five additional articles. Another example is how there are 
now well-developed scenarios on inter-group violence under different peace plans in Jerusalem down to 
the city level. Overall, computational models allow for more evidence-based and granular analysis. " Over 
the last five years, there has been a conflict event data revolution 

Global Data. Social Media and Field Experiments 

B) Global Database of Events, Language, and Tone (GDELT) is just coming online for accessing global data 
for conflict analysis. 

A] We are still evaluating 'IQ', which came online four months ago, for gaps. It is led by Stephen Shellman 
at the College of William and Mary. "SPEED" is not up to date and only includes data up until 2006. Uppsala 
Conflict Data Program (UCDP), Armed Conflict Location and Event Data project (ACLED), and the Social 
Conflict in Africa Database (SCAD) are mainly focused on Africa and not really anywhere beyond. Our 
proposal for our current projects included looking at 35 or so countries, but now we have coverage of more 
or less 50 countries. I know that ACLED does weekly or monthly Africa updates and I hope that in a few 
years, we will be able to have consistent worldwide updates. 

C) Two other trends include 1) using social media for conflict analysis, especially in the earliest stages of 
conflict; and 2) field experiments. We may be seeing a standard approach for field experimentation within 
political science. Jason Lyall [Director of the Political Violence FieldLab at Yale] and Chris Blattman have 
conducted field experiments (the latter mainly on economics). "Proving causal relationships is becoming 
more demanding in academia and experimental work can help make that case easier." 

A) Social media, like Facebook and Twitter, present other problems with the availability of data. The data 
usually available is incomplete and not vetted. Companies that aggregate social media traffic are making it 
less available and are increasing the prices; they are becoming more strict gatekeepers of data. It now 
means a high price for a small piece of data from self-selected participants and that leads it to not 
necessarily be great data despite it still being big data. 

B) Field experiments try to mimic lab environments. As social scientists we mostly often have to rely on 
random cases, like CERP. Since CERP was spent randomly by the military you can't really have or explain a 
control/treatment group. 

C) Field experiments can be used in post-conflict studies to look at interactions and to see what policies will 
work. 

A) There is also a big push on monitoring and evaluation and toward better metrics. For example, we can 
use 'matching', which is a statistical process, to FIND control variables and then backtrack experiments. 

C) For CERP we would look at what varies. We would look at villages which received CERP money and 
those that did not and compare the large set. We would match like villages with funding to similar and 
nearby villages without funding. 

A) It is increasingly harder to do matching at the granular level, especially over the long-term. 
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C) Our comparative advantage includes our resources and our training. 

A) Another advantage is that both our projects are 

Jhemim has j 3een p US hed through the i nteragency is even presented to 


(b)(3), (b)(6), (b)(7)(C) 


Presentations 


have also been made to the 


(b)(3), (b)(6). (b)(7)(C) 


Possible Collaboration 

C) What we would need is you [SIGAR] to articulate on paper the questions of analysis. This would allow us 
to estimate level of effort and time necessary for the analysis. 


B) We could, for example, study the effectiveness of CERP and make a presentation to you on it 





